Mobile electronic device with an adaptively responsive flexible display

ABSTRACT

A mobile electronic device having a flexible display device is disclosed. A request to retrieve information from a server over a wireless network is triggered based on flexing the flexible display device and other gestures.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/132,030 filed Apr. 18, 2016, which is a continuation of U.S. patentapplication Ser. No. 14/496,885, filed Sep. 25, 2014, which issued asU.S. Pat. No. 9,332,113 on May 3, 2016, which is a continuation of U.S.patent application Ser. No. 14/033,691 filed Sep. 23, 2013, which issuedas U.S. Pat. No. 8,855,727 on Oct. 7, 2014, which is a continuation ofU.S. patent application Ser. No. 13/775,135 filed Feb. 23, 2013, whichissued as U.S. Pat. No. 8,554,286 on Oct. 8, 2013, which is acontinuation of U.S. patent application Ser. No. 13/586,809 filed Aug.15, 2012, which issued as U.S. Pat. No. 8,396,517 on Mar. 12, 2013,which is a continuation of U.S. patent application Ser. No. 13/272,618filed Oct. 13, 2011, which issued as U.S. Pat. No. 8,346,319 on Jan. 1,2013, which is a continuation of U.S. patent application Ser. No.13/083,770 filed Apr. 11, 2011, which issued as U.S. Pat. No. 8,068,886on Nov. 29, 2011, which is a continuation of U.S. patent applicationSer. No. 12/185,406 filed Aug. 4, 2008, which issued as U.S. Pat. No.7,953,462 on May 31, 2011, the contents of which are all incorporated byreference herein as if fully set forth.

TECHNICAL FIELD

The present invention relates to an apparatus and method to communicatemultimedia documents or content to a mobile or fixed device over awireless network. In particular, the mobile or fixed device may beconfigured as a digital periodical or advertising device to transmit andreceive converted multimedia documents or content delivered usingmultimedia messaging service over a wireless network.

BACKGROUND

Wireless communications has enabled widespread easy access to mass mediaand multimedia information via mobile devices, cellular phones, personaldigital assistants, smart phones, or the like. Although the availabilityof multimedia information is ubiquitous, proper viewing, display, anddelivery of information for an enjoyable user experience is stilllacking. This is especially the case for viewing periodicals such asonline magazines or news content on a mobile device where a largedisplay is desirable. Moreover, having complex software and hardware isrequired to receive and display multimedia periodical information onpresent mobile devices making them expensive, heavy, and powerintensive.

Digital fixed advertising display systems are commonplace in urbanenvironments. However, widespread use of these systems is notcommonplace since they require a fixed power source and wired networkconnection. This is problematic for rural and remote locations such ashighways and light or utility posts where access to power andcommunications infrastructure is limited. Moreover, digital advertisingsystems do not provide any interactivity, location awareness, oraccessibility to Internet content making them poor for creatingimpressions on consumers and constraining their effectiveness.

Organic light emitting diode (OLED), liquid crystal display (LCD)flexible device technology, and other flexible substrate technologieshave made advancements recently promising to provide an enhanced userexperience, larger display sizes, portability, and low powerconsumption. However, commercialization of products having flexibledisplays has been slow due to a lack of viable and robust consumerapplications. Moreover, another problem with devices having flexibledisplays is the ability to integrate other electronic circuitry due toform factor constraints and poor design.

Multimedia Messaging Service (MMS) is a standardized technology beingdeveloped by the Third Generation Partnership Project (3GPP), the ThirdGeneration Partnership Project 2 (3GPP2), and the Open Mobile Alliance(OMA). Although originally developed to send photos and simple videoclips, recently added features to MMS have expanded its capabilities.The added features to MMS have made it into the preferable transport andpresentation mechanism of mobile messaging that rivals electronic mail.However, the commercial applications and utilization of advanced MMSfeatures has been limited. It is desirable to provide a digitalperiodical or advertising device preferably having a flexible displayfor receiving diverse mass media information, multimedia information, orInternet content in an efficient manner using MMS.

SUMMARY

An apparatus and method for delivering mass media and multimediainformation to a digital periodical or advertising device preferablyusing Multimedia Messaging Service (MMS) and having an electronicflexible screen or display is disclosed. A multimedia document, part ofa multimedia document, web page, multimedia advertisement, or any othermultimedia content on a server or a computer is extracted, parsed,and/or segmented into a plurality of video, audio, image, or textelements. The plurality of elements are each converted to an MMS messageand transmitted optionally with an MMS assembler message or embeddedinformation in each MMS message for proper reassembly of the originalmultimedia document or content. The MMS messages are received andreassembled back into the original multimedia document or content at adigital periodical or advertising device.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding of the invention may be had from thefollowing description, given by way of details and to be understood inconjunction with the accompanying drawings wherein:

FIG. 1 is a diagram of a digital periodical or advertising device inaccordance with an embodiment of the present invention;

FIG. 2 is a diagram of a digital periodical or advertising device havingan electronic flexible screen or display in accordance with anotherembodiment of the present invention;

FIG. 3 is a diagram of a system for providing and communicating massmedia documents, multimedia documents, or any multimedia content inaccordance with another embodiment of the present invention;

FIG. 4 is a diagram of a method for providing converted mass mediadocuments, multimedia documents, or any multimedia content usingMultimedia Messaging Service (MMS) in accordance with another embodimentof the present invention; and

FIG. 5 is a diagram of communicating converted mass media, multimediadocuments, or any multimedia content using a plurality of MMS messagesin accordance with another embodiment of the present invention.

DETAILED DESCRIPTION

The present invention will be described with reference to the drawingfigures wherein like numerals represent like elements throughout. FIG. 1is a diagram of a digital periodical or advertising device 100 formobile or fixed applications. Mobile applications include a digitalnewspaper, magazine, brochure, or any other digital mass media medium orapparatus. Fixed applications include a digital billboard, flyer or anyother advertising medium or apparatus.

Device 100 comprises computer bus 140 that couples one or moreprocessors 102, interface controller 104, memory 106 having software108, storage device 110, power source 112, display controller 120, oneor more displays 122, such as flexible organic light emitting diode(OLED) electronic displays, input/output (I/O) controller 116, I/Odevices 118, GPS device 114, one or more network adapters 128, and oneor more antennas 130. Device 100 may optionally have one or more motion,light, heat, radio frequency identification (RFID), face recognition,shape, or voice recognition sensors 126 and touch detector 124 fordetecting any touch inputs, including multi-touch inputs, for one ormore displays 122. Interface controller 104 communicates with touchdetector 124 and I/O controller 116 for determining user inputs todevice 100. Storage device 110 may be any disk based or solid statememory device for storing data. Power source 112 may be a battery, solarpanels for receiving and storing solar energy, or a device for receivingand storing wireless power as described in U.S. Pat. No. 7,027,311herein incorporated by reference as if fully set forth.

In device 100, one or more network adapters 128 may be configured as aTime Division Multiple Access (TDMA), Code Division Multiple Access(CDMA), Orthogonal Frequency-Division Multiplexing (OFDM), OrthogonalFrequency-Division Multiple Access (OFDMA), Global System for Mobile(GSM) communications, Enhanced Data rates for GSM Evolution (EDGE),General Packet Radio Service (GPRS). cdma2000, Wideband CDMA (W-CDMA),Long Term Evolution (LTE), 802.11x, Wi-Max, mobile Wi-MAX, or any otherwireless or wired transceiver for modulating and demodulatinginformation communicated via one or more antennas 130.

In one embodiment of the present invention, one or more network adapters128 are configured to only handle data traffic thereby eliminating theneed for circuitry and software for handling voice traffic. This isadvantageous as the circuitry for handling and processing voice trafficin mobile devices occupies substantial space, adds complexity, andresults in unneeded cost for a digital periodical device where voicecapability may be unnecessary.

FIG. 2 is a diagram of a digital periodical or advertising device 200having one or more electronic flexible screen or display devices 202.The one or more electronic flexible screen or display devices 202 may beconfigured, manufactured, produced, or assembled based on thedescriptions provided in U.S. Patent Publication Nos. 2007-247422,2007-139391, 2007-085838, or 2006-096392 or U.S. Pat. No. 7,050,835 orWO Publication No. 2007-012899 all herein incorporated by reference asif fully set forth. The one or more electronic flexible screen ordisplay devices 202 may be configured and assembled using organic lightemitting diodes (OLED), liquid crystal displays using flexible substratetechnology, flexible transistors, field emission displays (FED) usingflexible substrate technology, as desired. In the case for multipleelectronic flexible screens or display devices 202, rotation 214 revealsa second electronic flexible screen or display device 203 on the back ofdevice 200.

Any one of housing members 204 and 206 selectively house certainhardware components described in device 100 such as one or moreprocessors 102, memory 106, one or more sensors 126, and one or morenetwork adapters 128. In one embodiment of the invention, housingmembers 204 and 206 may each have individual transmitting/receivingantennas 216 and 218 for providing spatial and time diversity. One ormore electronic flexible screen or display devices 202 can optionally beconfigured to collapse 208 and roll up into housing members 204 or 206for portability. For orientating device 200 in a circular manner for apresentation or posting on a lightpost as an advertisement, attachmentmembers 210 and 212 may be used. Housing members 204 and 206 may berectangular or cylindrical and provide grasping positions, the abilityto position device 200 upright on a flat surface, or the ability tomount device 200 on a flat surface, as desired. It is appreciated by oneskilled in the art that housing members 204 and 206 and one or moreelectronic flexible screen or display devices 202 are not drawn toscale. They may have different shapes and dimensions while configured toprovide the same functionality provided herewith.

Still referring to FIG. 2, in another embodiment device 200 mayoptionally have a liquid crystal display (LCD), LED, FED, or OLEDdisplay unit 220. For this case, when one or more electronic flexiblescreen or display devices 202 is collapsed into housing member 206adjacent display unit 220 is still available for displaying content.When one or more electronic flexible screen or display devices 202 isexpanded out of housing member 204 or 206, the combination of displayunit 220 and flexible screen or display devices 202 provides a largerscreen size for a single graphical feed or for having separate graphicalfeeds or windows in each display unit, as desired. In thisconfiguration, the images displayed on flexible screen or displaydevices 202 can be responsive to one or more sensors 126 detecting abending of flexible screen or display devices 202.

FIG. 3 is a diagram of a system for providing and communicating massmedia documents, multimedia documents, or any multimedia content inaccordance with an embodiment of the present invention. Server orcomputer 300 may be configured as a web server, a multimedia messagingservice center (MMSC), an MMS relay server, or a personal computer, asdesired. Server or computer 300 at least comprises memory 302 coupled toone or more processors 310, one or more network adapters 312, andwireless power transmitter 320 via computer bus 305. One or more networkadapters 312 may be configured to implement any one of wireless or wiredtechnologies given above. Wireless power transmitter 320 may beconfigured to provide power to device 318 as described in U.S. Pat. No.7,027,311 herein incorporated by reference as if fully set forth.

Memory 302 comprises a converter engine 304 having documents or content306 and 308 that may be in a Microsoft WORD format, a Microsoft EXCELformat including graphs, a Portable Document Format (PDF), a PDFdocument having embedded multimedia advertisements including video andaudio, an update for a PDF document previously transmitted for updatingembedded multimedia advertisements including video and audio, aHypertext Markup Language (HTML) format, a Microsoft PowerPointpresentation format, a digital multimedia periodical document, a digitalperiodical document having embedded multimedia advertisements, a digitaladvertising format, part of a digital advertising format, a selectedsubset of a multimedia document or the like having multimediainformation. Converter engine 304 extracts, parses, and/or segmentsdocument 306 by detecting one or more video elements 306 ₁, one or moreaudio elements 306 ₂, one or more image elements 306 ₃, or one or moretext elements 306 ₄. The video elements 3061 may be flash video,shockwave video, MPEG video, streaming video, Advanced Video Coding(AVC), H.264 video, or the like. The audio elements 306 ₂ may be an MP3,WAV, Advanced Audio Coding (AAC), streaming audio, or the like. Theimage elements 306 ₃ may be JPEG, TIFF, bitmap, or the like.

One or more text elements 306 ₄ may represent the body of document 306that is divided into multiple parts. Any one of elements 306 ₁-306 ₄ mayalso represent dynamic advertising information, such as video or audio,that is embedded into document 306. The dynamic advertising informationmay be updated based on user preferences, a target audience, location,or the like. In an embodiment of the present invention, document 306 isa PDF document and elements 306 ₁-306 ₁ sent via MMS are updates tomultimedia advertising preexisting in the PDF document that are PUSHEDto user agent 316 by computer or server 300.

Multimedia Messaging Service (MMS) is capable of communicating at leastthe following types of content:

1. Short Message Service (SMS) types;

2. plain text;

3. speech;

4. audio;

5. synthetic audio, such as Scalable Polyphony MIDI;

6. still images, such as JPEG;

7. bitmap graphics;

8. video, including streaming video;

9. vector graphics, such as Scalable Vector Graphics (SVG), Tiny 1.2, orECMAScript;

10. File Format for dynamic media;

11. Media synchronization and presentation format;

12. Digital Rights Management (DRM) format; and

13. Extensible Markup Language (XML).

Depending on the content of document 306, elements in addition to 306 ₁to 306 ₄ may be formed based on the types supported above by MMS. Oncedocument 306 is extracted, parsed, and/or segmented into the appropriatenumber of elements representing its contents, each element 306 ₁ to 306₄ is converted into individual MMS messages with the recipient addressof MMS user agent 316 by converter engine 304. Elements 306 ₁ to 306 ₄are then transmitted over network 314 as individual MMS messages todevice 318 having MMS user agent 316. Device 318 may be configured asdevice 100 or 200, as desired. MMS user agent 316, which may be executedby software running on one or more processors 102, reassembles document306 using elements 306 ₁ to 306 ₄ received as individual MMS messages.

Reassembly may be performed selectively by loading different parts ofdocument 306 as the individual MMS messages are received by device 318.Moreover, an MMS assembler message may optionally be transmitted byserver or computer 300 having information required to properlyreassemble elements 306 ₁ to 306 ₄ back into document or content 306.The assembler may be an individual Synchronized Multimedia IntegrationLanguage (SMIL) or Wireless Markup Language (WML) message thatcoordinates and explains the relationship between elements 306 ₁ to 306₄ so that it can be reassembled back into document or content 306.Alternatively, each MMS message is embedded with individual information,such as SMIL or WML language, describing the correct positioning orplacement, including page number in the case of a multi-page document,and relationship to other elements within document or content 306 todisplay the delivered information. In an embodiment of the presentinvention, each received element received by device 318 may bepositioned on one or more display devices 122 or 202 based on thephysical position and viewing angle of one or more users detected by oneor more sensors 126.

Still referring to FIG. 3, in one embodiment of the present inventionelements 306 ₁ to 306 ₄ may delivered over network 314 using an InternetProtocol (IP) Multimedia Subsystem (IMS). Using IMS, the elements 306 ₁to 306 ₄ may be packetized into IP packets by converter engine 304 sothat they bypass routing through an MMSC and delivered directly to MMSuser agent 316. For the case where MMS user agent 316 is in a mobilephone configured for end-to-end IP communication, elements 306 ₁ to 306₄ may be routed using an IP address included in each MMS message ratheran International Mobile Subscriber Identity (IMSI).

FIG. 4 is a diagram of method 400 for providing converted mass mediadocuments, multimedia documents, or any multimedia content usingMultimedia Messaging Service (MMS). A trigger event (402) causes theretrieval (404) of documents or content to memory 302. Trigger event(402) may be a scheduled PUSH of information at a predetermined time,when a news story is generated, a user request for a multimediadocument, a request from device 100 for a multimedia document includingas a result of feedback received by one or more sensors 126, a requestfrom device 200 for a multimedia document including as a result offeedback received by any one of sensors 126, or an initiation by serveror computer 300.

Trigger event (402) may also be a change in the shape or bending of oneor more electronic flexible screen or display devices 202 detected bythe plurality of sensors 126 that generates a request for documents fromserver or computer 300. For this case, documents or content such as htmlor web content in addition to MMS messages can be delivered to device318 responsive to the detected change of shape of one or more flexiblescreen or display devices 202. This may be done by embedding one or moresensors 126 in one or more flexible screen or display devices 202 orhousing members 204 or 206 that detect a change in gradients of theflexible display.

In one embodiment of the present invention, the retrieval of documentsor content (404) is triggered by initiating a “PRINT to MMS” function, amouse click function requesting “SEND by MMS”, or a “SEND MULTIMEDIAONLY by MMS” function on server or computer 300 when configured as apersonal computer that can be integrated into a single personal computerapplication, preferably without the need of any other third partysoftware, and information transported by configuring one or more networkadapters 312 as a cellular transceiver or any other wireless or wiredcommunication technology to MMS user agent 316. A personal computerapplication may be a web browser where clicking on a link to download adocument or a subset of a document also provides the option to downloadand send the content via a plurality of segments or elements in aplurality of MMS messages. In one embodiment, each multimedia element inthe document that is compatible for transport via MMS may have anindicator, such as an icon, so that only compatible multimedia elementsare selected for transport.

Having MMS capabilities on a personal computer enables easy transportand communication of information between a single personal computerapplication on computer 300 and device 318. Communication of personalcomputer content using MMS is advantageous since the number of devicessupporting MMS is expected to outgrow the number of devices supportingmobile electronic mail.

For the “SEND MULTIMEDIA ONLY by MMS” function, photos, audio, and videoare sent from the original document or content without the text in thedocument or content to the intended recipient. This is advantageous if arecipient only desires to see the multimedia content in the originaldocument or content. When the MMS messages having elements 306 ₁ to 306₃ with only the multimedia content of the original document is received,device 100 or 200 may display the multimedia elements in a list or acascade based on importance, size, date, quality, type, or any othercriteria. If any of the MMS messages represent an advertisement in theoriginal document, the placement of the advertisement on the displays ofdevice 100 or 200 may be based on creating the greatest number ofimpressions to the user such as on the top, left hand side, or righthand side. Alternatively, received elements 306 ₁ to 306 ₃ may be placedin the same position on the displays of device 100 or 200 as theyexisted in the original document when received.

Referring again to device 100, trigger event (402) may also be GPSdevice 114 detecting a change in position in the case where device 100is mobile. Using GPS, elements 306 ₁ to 306 ₄ may be transcoded intodifferent audio, image, and video qualities if server or computer 300determines that device 318 configured as device 100 is in a building,behind heavy foliage, in a tunnel, approaching an unfavorable positionin the network, or the like. GPS device 114 can also be used by serveror computer 300 to determine when to push a particular advertisementdepending on the location of device 318. This is advantageous forproviding localized advertising information.

For the case that power source 112 receives wireless power from wirelesspower transmitter 320, trigger event (402) may be a change in theproximity to a wireless power coverage area requiring transcoding ofsubsequent elements 306 ₁ to 306 ₄. If the wireless power coverage areais strong, elements 306 ₁ to 306 ₄ may be transcoded into higher qualityaudio, image, and video since the required increased processing powercan be compensated for by the wireless power transmitter 320. If thewireless power coverage area is weak elements 306 ₁ to 306 ₄ may betranscoded into lower quality audio, image, and video in order toconserve power.

Referring again to device 200, trigger event (402) may also be caused ondevice 200 by one or more sensors 126 detecting motion such as rotation.When device 200 is rotated 214, content is loaded on the secondelectronic flexible screen or display device 203 and the one or moreelectronic flexible screen or display devices 202 is turned off. Anothertrigger event (402) in accordance with the present invention is when oneor more sensors 126 detect a bending of one or more electronic flexiblescreen or display devices 202 indicating that the device has been foldedor its shape altered, such as made circular, cubicle, or the like. Ifflexible screen or display devices 202 is folded or bent, the deliveredMMS messages received by device 200 will be altered or resized to fitthe new shape. For each trigger event (402), a request is sent tocomputer 200 for a multimedia document or content subsequent to thetrigger event on device 200.

Subsequent to retrieval (404), the document or content is extracted,parsed, and/or segmented into a plurality of elements 306 ₁ to 306 ₄ andeach converted into individual MMS messages (406) by converter engine304. Each element 306 ₁ to 306 ₄ comprises an individual video, image,audio, or text information of the document or content. Optionally, anMMS assembler message may then be generated (408) in order for therecipient MMS user agent 316 to know how to properly reassemble thereceived elements. Rather than an independent MMS assembler message, theposition and placement information within the original document orcontent of the information delivered in each MMS message may be embeddedin the header or body of the MMS message for proper reassembly.Alternatively, a template of a page or publication may be stored ondevice 100 and used to properly reassemble all the transmitted andreceived MMS elements into the original document.

Elements 306 ₁ to 306 ₄ representing the original document or contentare then transmitted over network 314 using a plurality of MMS messageswith optionally the MMS assembler message (410) to MMS user agent 316.The MMS messages are then received optionally with the MMS assemblermessage (412). The original document or content is then reassembledoptionally using the MMS assembler message or embedded information(414).

FIG. 5 is a diagram of communicating converted mass media, multimediadocuments, or any multimedia content using a plurality of MMS messages.A multimedia document or content 502 is either extracted, parsed, and/orsegmented into image element 502 ₁, audio element 502 ₂, text element502 ₃, and video element 502 ₄ and an MMS message is generated for eachelement. An MMS assembler message 504 may optionally be generated havinginformation to properly reassemble converted document or content 502.Alternatively, each MMS message having any one of elements 502 ₁ to 502₄ may have embedded information for reassembling the converted documentor content 502. The embedded information includes placement orpositioning information of the image, audio, text, or video elementwithin the original document or content. MMS messages 506 ₁-506 ₄ andoptionally MMS assembler message 504 is communicated over network 507 inany order. MMS messages 502 ₁-502 ₄ are received as elements 510 ₁-510 ₄along with MMS assembler message 508. The original multimedia documentor content 502 is reassembled as each element is received and displayedin segments.

Although the features and elements of the present invention aredescribed in the preferred embodiments in particular combinations, eachfeature or element can be used alone without the other features andelements of the preferred embodiments or in various combinations with orwithout other features and elements of the present invention. Thepresent invention may be implemented in a computer program tangiblyembodied in a computer-readable storage medium for execution by aprocessor or a general purpose computer for use with or by anynon-volatile memory device. Suitable processors include, by way ofexample, both general and special purpose processors. Typically, aprocessor will receive instructions and data from a ROM, a random accessmemory (RAM), and/or a storage device. Storage devices suitable forembodying computer program instructions and data include all forms ofnon-volatile memory, including by way of example semiconductor memorydevices, magnetic media such as internal hard disks and removable disks,magneto-optical media, and optical media such as CD-ROM disks andDigital Versatile Disks (DVDs). Types of hardware components orprocessors which may be used by or in conjunction with the presentinvention include Application Specific Integrated Circuits (ASICs),Field Programmable Gate Arrays (FPGAs), microprocessors, or anyintegrated circuit.

What is claimed is:
 1. An electronic device comprising: a flexibledisplay; a plurality of sensors configured to detect folding of theflexible display; and circuitry configured to resize an image displayedon the flexible display in response to the plurality of sensorsdetecting folding of the flexible display.
 2. The electronic devicedefined in claim 1, wherein the plurality of sensors are configured todetect folding of the flexible display into a given shape and whereinthe circuitry is configured to resize the image displayed on theflexible display based on the given shape.
 3. The electronic devicedefined in claim 1, wherein the plurality of sensors are configured todetect folding of the flexible display into a given shape and whereinthe circuitry is configured to resize the image displayed on theflexible display to fit the given shape.
 4. The electronic devicedefined in claim 1, wherein the plurality of sensors are configured todetect that a shape of the flexible display has been altered and whereinthe circuitry is configured to resize the image displayed on theflexible display in response to the plurality of sensors detecting thatthe shape of the flexible display has been altered.
 5. The electronicdevice defined in claim 4, wherein the circuitry is configured to resizethe image displayed on the flexible display to fit the shape of theflexible display in response to the plurality of sensors detecting thatthe shape of the flexible display has been altered.
 6. The electronicdevice defined in claim 1, wherein the flexible display comprises anorganic light-emitting diode flexible display.